The rapid expansion of information service and data processing industries has resulted in a need for computer systems to manage and store large amounts of data. As an example, financial service industry businesses such as banks, mutual fund companies or the like often operate complex data processing systems that require access to many hundreds of gigabytes or even terabytes of data. Data storage system developers have responded to these types of data storage requirements by integrating large capacity data storage systems, data communications devices and computer systems into networks called “storage networks” or “storage area networks” (SANs.) A storage area network is a collection of data storage systems that are networked with a number of host computer systems that operate as servers to access data stored in the data storage systems. Components of a typical conventional storage area network implementation include one or more connectivity devices such as high speed data switches that interconnect the various data storage systems to each other and to one or more host or server computer systems (servers) that require access to (e.g., read and/or write) the data in the data storage systems on behalf of client software applications and/or client computer systems to communicate with the servers. A SAN is thus a fabric that routes input/output requests in data access protocols such as SCSI, iSCSI, Fiberchannel, or others from servers through switches to one or more data storage systems for access to data stored in those storage systems.
A developer or administrator of a storage area network environment may install one or more distributed storage area network management software applications within the storage area network to manage or administer the various resources (i.e., devices, computer systems, storage systems, etc.) that operate within the storage area network. A network manager (i.e., a person) responsible for management of the storage area network operates the network management software application to perform management tasks such as performance monitoring, network analysis, remote configuration, storage provisioning and other administration of the various resources operating within the storage area network.
A typical conventional storage area network management software application may have several different software components or processes that execute independently of each other in a distributed manner on different computer systems but that collectively interoperate together to perform network management. As an example, conventional designs of storage area network management applications can include console, server, agent and storage or database management software components or processes that each performs a specific role in storage area network management.
Generally, the server component or process operates as a central control process within the storage area network management application and coordinates communication between the console, storage and agent components. The console component or process often executes within a dedicated storage area network management workstation to allow a network administrator (i.e., a person or people responsible for management of the storage area network) to visualize and remotely control and manage the various components within the storage area network that are displayed on a graphical user interface within a display on the console computer system. Agent components or processes execute on the various host computer systems such as servers distributed throughout the storage area network to manage various types of storage area network resources.
As an example, there may be different respective agents specifically designed (e.g., coded in software) to remotely manage, control and collect data from data storage system resources, database applications, switches, and so forth. Agents receive remote management commands from the server (i.e., via a user at the console controlling the server) and apply functionality associated with those management commands to the managed resources within the storage area network for which those agents are designated to manage. Agents are also responsible for periodically (e.g. nightly) collecting configuration or management data concerning the storage area network resources that those agents are responsible for managing. As an example, an agent that is responsible for monitoring and control of a large data storage disk array system might be configured to collect management data from that storage array on a nightly basis. This agent data collection process might include communicating with the storage array to query the array for information concerning all devices (e.g. logical storage devices) configured at that time within the array. This may result in the collection of large amounts of management data from certain managed SAN resources, such as large storage array in which there can be many thousands of devices configured for operation. The management data collected from the agent is also known as agent data.
Agents transmit this collected agent data back to an assigned storage component, also known as a store process. The store process receives the collected agent data from the agent and processes and stores this information into a storage area network management database for access by the server and console components. The management database thus provides managed object data (i.e., agent data that has been stored in the management database) for all managed resources in the SAN. The console component can interact with the server component to obtain current network information such as performance, capacity, load or other data concerning managed resources within the storage area network by accessing the managed object data maintained in this manner within the network management database.
The design of conventional network management applications often uses a large number of agent components that each remotely operate to periodically collect sets of management data from the managed resources that those agents manage. After collection, each agent is responsible for transmitting the collected set of management data to a store process that processes the management data for storage into a management database or other accessible management data repository. More specifically, an agent responsible to interact with that managed resource collects and forwards management data associated with that data storage system to the store process for placement into the management database. The store process receives the entire set of collected management data and converts each field of this data from one data type or arrangement to another and then places this data into the network management database.
A data management framework provides an efficient method for processing and storing network management data collected from a managed resource by an agent. The framework provides a set of adapter classes that allow store processing to efficiently process agent data collected from managed resources. During operation of the store process disclosed herein, the store process obtains agent data collected by an agent from a managed resource in a network and populates a set of adapters with the agent data. The store process determines the set of adapters to be instantiated from the agent data collected from the managed resource. In particular, the store process identifies and instantiates at least one type adapter associated with the agent data collected from the managed resource. As an example, the type adapter may correspond to a resource such as a storage array from which the agent data was collected.
For each type adapter, the framework provides instance adapters and the store process instantiates at least one associated instance adapter to contain agent data associated with that type adapter. In this manner, instance adapters operate as containers for the collected management data and are associated to type adapters that correspond to managed resources from which that agent data was collected. The store process can validate the data within the instance adapters to be certain that the data is complete for processing into a managed object database access by the network management application. By performing validation, the store process ensures that cumbersome database access operations are avoided in situations when agent data is corrupted or incomplete in some manner.
Assuming the validation succeeds, the store process begins the operation of storing the agent data as managed object data within the management database maintained by the network management application. To do so, based on the set of adapters populated with the agent data, the store process selects and applies at least one of a plurality of read mechanisms to read corresponding managed object data from a management database into managed objects that are to be synchronized with the agent data collected from the managed resource. To be most efficient, the store process selects a read mechanism that reads only managed object data that corresponds to the agent data.
Using the selected read mechanism, the store process reads from the management database and stores the managed object data read from the management database using the invoked read mechanism in a managed object structure for comparison with the agent data maintained in the instance adapters. Using the agent data being maintained in the instance adapters and the managed object data being maintained in memory from application of the read mechanism, the store process thereafter synchronizes the agent data populated in the set of adapters with the managed object data in the management database.
The store process can create new managed objects for agent data stored in instance adapters that does not have corresponding managed object data read from the management database. Likewise, the store process deletes managed objects for which managed object data was read from the management database but for which there is no corresponding agent data maintained in at least one instance adapter. The store process also associates specific instance adapters containing agent data that map to specific managed objects read from the management database using key information that is common between the agent data and object data read from the database. Once instance adapters are matched to managed object structures, the store process copies properties of each instance adapter that maps to a specific managed object into corresponding properties of that managed object. Thereafter, the store process commits the managed object data in the managed objects to the management database. Agent designers and developers create an agent data structure, from scratch, that will be used to transfer of management data collected by the agent (i.e., agent data) for transmission to a store process. Likewise, the store process itself must be modified to be able to processes each newly created agent data structure for each new type of networked managed resource (e.g. each new device that must be managed in a network).